Skip to content

Add task to collect proto files and add steps to release them with tester, nightly and stable builds #1931

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Oct 25, 2022

Conversation

silvanocerza
Copy link
Contributor

Please check if the PR fulfills these requirements

See how to contribute

  • The PR has no duplicates (please search among the Pull Requests
    before creating one)
  • The PR follows
    our contributing guidelines
  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been added / updated (for bug fixes / features)
  • UPGRADING.md has been updated with a migration guide (for breaking changes)

What kind of change does this PR introduce?

Adds a new task and changes the tester, nightly and stable build release processes.

What is the current behavior?

Currently there is no easy way for developers to download only .proto files to work on projects that depend on them.
Those files are necessary to generate other languages files to handle gRPC communication.
The only way as of now is to download the whole arduino-cli repo and copy the files.

What is the new behavior?

A new command has been added to zip all the .proto files and save them in dist, the task is called protoc:collect.
The output zip will be called arduino-cli_<version>_proto.zip, <version> as defined here. The command can be run locally and on CI.

This zip will also be included in the released files for tester, nightly and stable builds.

Does this PR introduce a breaking change, and is titled accordingly?

None.

Other information

Closes #588.

@silvanocerza
Copy link
Contributor Author

@cmaglie cmaglie self-assigned this Oct 20, 2022
@cmaglie cmaglie added type: enhancement Proposed improvement priority: medium Resolution is a medium priority topic: infrastructure Related to project infrastructure criticality: low Of low impact labels Oct 20, 2022
Comment on lines 164 to 168
# Run this after checksums or the uploaded proto files zip
# will make the checksum job always fail
needs:
- checksums
- package-name-prefix
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why checksums is failing? shouldn't just add the checksum of the protoc archive to the checksums file together with the other archives?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't think about that, it would make sense since it's a file part of the release. 🤔
I'd have to edit a bit the checksum process I think, I'll take a look at it.

Comment on lines 135 to 163
proto-files:
needs:
- package-name-prefix
runs-on: ubuntu-latest

steps:
- name: Checkout repository
uses: actions/checkout@v3
with:
fetch-depth: 0

- name: Install Task
uses: arduino/setup-task@v1
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
version: 3.x

- name: Collect proto files
run: |
PACKAGE_NAME_PREFIX=${{ needs.package-name-prefix.outputs.prefix }}
export PACKAGE_NAME_PREFIX
task protoc:collect

- name: Upload proto files zip
uses: actions/upload-artifact@v3
with:
path: ${{ env.DIST_DIR }}/*.zip
name: rpc-protocol-files

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you think about adding this as an additional step in the matrix of build job? It seems like almost all the steps are the same, we could avoid some code duplication.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's semantically correct, this job doesn't build anything in itself.
It also would require some changes to that job to make it more generic if we want to go toward that direction too.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't duplicate the code that way. There are some things that could be made in order to achieve that:

  • If we move the task protoc:collect to DistTask.yml the workflow do not require any big change.
  • another idea would be to simply rename the task as dist:protoc-collect
  • or maybe just move dist: in the matrix.os.task

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wasn't DistTask.ymlshared in other projects too? I thought of putting it there but didn't cause of that.

Second and third might work but we'd still need to reword the build job and os matrix variable to be generic.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mhh yes, the DistTask.yaml is shared and probably better to not touch it... Second and third seems more approachable to me with the proper rewording

@umbynos umbynos merged commit 13f2255 into arduino:master Oct 25, 2022
@silvanocerza silvanocerza deleted the protoc-collection branch October 25, 2022 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
criticality: low Of low impact priority: medium Resolution is a medium priority topic: infrastructure Related to project infrastructure type: enhancement Proposed improvement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[proposal] Release the proto files with the CLI
3 participants